Repository navigation
feat: package Altimate for the Codex directory with guided setup - #9
Conversation
Code Review SummaryStatus: 2 Issues Found | Recommendation: Address before merge Incremental pass over Overview
Issue Details (click to expand)SUGGESTION
Files Reviewed (11 files)
Reviewed with targeted cross-checks (not PR-diff scope): Fix these issues in Kilo Cloud Previous Review Summaries (2 snapshots, latest commit 0f67a8e)Current summary above is authoritative. Previous snapshots are kept for context only. Previous review (commit 0f67a8e)Status: No Issues Found | Recommendation: Merge This pass reviewed only the incremental diff Files Reviewed (4 files)
Previous review (commit 4b065a3)Status: 8 Issues Found | Recommendation: Address before merge Overview
Issue Details (click to expand)CRITICAL
WARNING
SUGGESTION
Fix these issues in Kilo Cloud Files Reviewed (13 files)
The builder's allowlist approach, symlink and unsupported-file rejection, fixed-timestamp reproducibility, and manifest stripping are sound; the packaging claims in Reviewed by deepseek-v4.1-flash-1017 · Input: 89.3K · Output: 31K · Cached: 969.2K |
|
Addressed the eight bot items in 9b224e0 and 0f67a8e. Fixed MCP env/path validation, malformed metadata diagnostics, the README archive test, resource-reference documentation and the skill count. Kept the scoped npm name (both package names currently publish 0.12.11), the distinct-artifact-path assertion and the artifact-local CI gate, with explanations in each thread. All 12 packaging tests, existing resolver/hook suites, real ZIP build and diff checks pass. Independent Sol/Terra reviews completed; the additional dot-relative path finding was fixed and independently rechecked with 36 path cases. Clean desktop install, distribution switching and OpenAI approval remain unverified. No merge performed. |
sahrizvi
left a comment
There was a problem hiding this comment.
Consensus review. Panel: Claude (lead) and Codex. Both reviewers raised both majors and both test minors on their own.
Verdict: changes requested. Two major findings are open.
What I checked at 0f67a8ec:
- Built the default ZIP twice. Both builds give the same SHA-256 (
232bcf59…), and so do builds with Python 3.9, 3.13 and 3.14. The archive has 11 entries:.codex-plugin/plugin.json,LICENSE,PRIVACY.md,README.md(=DIRECTORY.md),SECURITY.md,TERMS.md,assets/{logo,composer-icon}.svgandskills/{altimate-code,get-started,review}/SKILL.md. Every entry has a fixed 2020-01-01 timestamp, mode 0644 and sorted order. - Nothing else is in the archive: no hooks, no commands, no app or MCP registration, no
.git, tests, CI files or local paths. The skill files match the GitHub plugin byte for byte. The GitHub plugin keepshooks/hooks.json(SessionStart, UserPromptSubmit, Stop). - The manifest is 1.3.0, which matches CHANGELOG.
defaultPromptlengths are 57, 44 and 22.shortDescriptionis 27 characters.requires_local_executorsits at the top level, andextensions.com.openai.onboardingSkilluses the placement the OpenAI desktop-only and submission pages give. - The listing SVGs match
altimate_logo.svgat the named frontend revision, whose SHA-256 matches DIRECTORY.md. Only the opening<svg>tag differs. - The CI workflow runs on
pull_request/push/workflow_dispatchwithcontents: readand no secrets. It uploads onlydist/*.zipas a workflow artifact. A fork PR gets no write access. - Tests:
tests/*.shpass at base and head, andpytest tests/at head gives 12 passed.
Remaining validation: the ZIP has never been installed in desktop Codex, and the hooks and the directory-to-GitHub switch have never been run. That blocks submission, not merge, because DIRECTORY.md already says to test a clean install before uploading. The majors below are merge blockers because get-started ships in the GitHub plugin as soon as this merges.
Inline comments carry each finding. The nits are in a separate PR comment.
|
Consensus review nits (panel: Claude and Codex; both accepted all four). None of them blocks.
Already covered by the author's replies, so not raised again: both npm package names (both resolve to the same release) and the duplicate test run in the packaging workflow. |
0f67a8e to
1736c27
Compare
|
Addressed review5934977758 and the nits in this comment, pushed in 1736c27:
All 14 packaging tests, CLI resolver and workspace-hook suites, the real ZIP build and diff checks pass. Mutation checks detect stripping, timestamp, permissions, order and CRLF artwork regressions. Fresh independent GPT-6 Astra and GPT-6.1 Sol reviews both explicitly APPROVED the full rebased PR at 1736c27 before push. A supplemental Sol security pass also approved (102 invalid-input probes, three valid configurations, 13 symlink cases). No Claude reviewer was used. Live desktop installation, hook switching and OpenAI submission acceptance remain untested and are still required before submission. Inline replies explain each fix. Please re-review the updated head; this comment does not change the existing human review verdict. |
| def test_reviewed_mcp_bypasses_rejected(self): | ||
| values = ["server/../../../../Users/me/x.js", "--config=cfg/../../etc/secret", | ||
| "-c/Users/me/cfg.json", "-javaagent:/Users/me/a.jar", | ||
| "file:///Users/me/srv.js", "$HOME/srv/index.js", "${HOME}/srv/index.js", |
There was a problem hiding this comment.
SUGGESTION: Colon/URI rejection is not pinned by these fixtures
file:///Users/me/srv.js is rejected by the empty-path-component check (token.split("/") contains ""), not by the character class that forbids :, and -javaagent:/Users/me/a.jar is rejected earlier by the option-name regex. Adding : to the allowed character class in validate_mcp_token (scripts/build_directory_zip.py:48) therefore keeps the whole suite green, while --config=C:/Users/me/config.json (a drive-letter absolute path) would then be accepted and written into the packaged .mcp.json. Add one fixture such as "--config=C:/Users/me/config.json" to pin the colon rule directly.
| values = ["server/../../../../Users/me/x.js", "--config=cfg/../../etc/secret", | ||
| "-c/Users/me/cfg.json", "-javaagent:/Users/me/a.jar", | ||
| "file:///Users/me/srv.js", "$HOME/srv/index.js", "${HOME}/srv/index.js", | ||
| "--token=sk-live-synthetic", "--token", "${PLUGIN_ROOT}/../secret", |
There was a problem hiding this comment.
SUGGESTION: Case-insensitive credential rejection is untested
Every credential fixture here is lowercase, so dropping re.I from re.search(r"token|secret|password|credential|api-?key|authorization", option, re.I) (scripts/build_directory_zip.py:36) leaves all tests passing, while --TOKEN=sk-live-... would pass the grammar and be written verbatim into the packaged .mcp.json. Add an uppercase variant (e.g. "--TOKEN=sk-live-synthetic") to this list.
sahrizvi
left a comment
There was a problem hiding this comment.
Approve. Re-review of 1736c2767d by a two-model panel: Claude (lead) and Codex (default model, read-only).
Both round-1 majors are fixed:
get-startedinstaller. The model may run onlynpm install -g @altimateai/altimate-code, and only when the user asks. Thecurl … | bashinstaller is now something the user runs in their own terminal. The skill forbids running it, or downloading and running its script, through a tool call. Authentication is still a terminal handoff, and the skill says never to ask for tokens in chat.- Local-MCP path checks. Every bypass from round 1 is now rejected, plus
~/,./, bare..,${PLUGIN_ROOT}/../x, absolutecommand/cwd, Windows paths andkey=/abs. The error messages never repeat the rejected value. A symlinked.mcp.json, skill file or asset directory is rejected too. All 24 of my probe inputs were rejected.
What I checked:
- Tests.
tests/*.shandpython3 -I -m pytest tests/pass at head (14 passed, 42 subtests). At base, the shell tests pass and pytest finds no tests. - Mutations. I made 10 changes to the builder and assets. The tests caught every one:
- removing the
..component check - widening the option regex
- widening the value regex
- removing the literal-env check
- removing the field allow-list
- removing the credential-option check
- dropping
hooksfrom the strip tuple - dropping
appsfrom the strip tuple - using clock timestamps
- giving
logo.svgCRLF line endings
- removing the
- Default ZIP. It has 11 files and is byte-identical across two builds. It contains no hook, command, app reference, MCP registration,
.gitdata, test, CI file or local path. The task skills match the GitHub plugin's files byte for byte and are unchanged from base. The GitHub plugin still has its SessionStart, UserPromptSubmit and Stop hooks. - Manifest. Version 1.4.0 matches the CHANGELOG.
defaultPromptlengths are 57, 44 and 22 characters. - Artwork. The icons match the named frontend revision (SHA-256
d643aeef…) except for the outer<svg>tag. - Workflows. Both use
pull_requestwithcontents: read, so forks get no write access. The packaging actions are pinned to SHAs that match their v4, v5 and v4 tags.
Four minors remain, inline. None of them blocks:
- Some credential-named options get past the denylist (
--auth=,--passwd,--access-key=). Edge input: this only affects the opt-in local-MCP build, and DIRECTORY.md already says to review arguments for secrets by hand. - MCP server names are not checked. Edge input, opt-in build only. Safe in a follow-up.
- Symlinks are only checked on the last part of a path, not on parent directories. Edge input: it needs a symlinked directory committed to this repo. Safe in a follow-up.
- No test protects the
get-startedinstaller rules. Test-only.
The ZIP has still not been installed in desktop Codex. That blocks submission, not merge, and DIRECTORY.md already lists it before upload.
| option, separator, token = value.partition("=") | ||
| if not re.fullmatch(r"--?[A-Za-z][A-Za-z0-9-]*", option): | ||
| raise ValueError("MCP options require separate values or --option=value syntax") | ||
| if re.search(r"token|secret|password|credential|api-?key|authorization", option, re.I): |
There was a problem hiding this comment.
Minor (edge input, opt-in local-MCP build only, safe in a follow-up). The credential-option denylist misses common names. {"command":"example","args":["--auth=x1"]}, ["--passwd","x2"] and ["--access-key=AKIAX"] all pass validate_local_mcp and are written to .mcp.json unchanged (reproduced). passwd doesn't match password, access-key doesn't match api-?key, and auth doesn't match authorization. DIRECTORY.md does say this is not a secret scanner, so this is a gap in a best-effort check, not a broken contract. Suggested fix: widen the pattern (auth|passw|pwd|key|bearer|session|cookie), or reject every --x=value whose name isn't on a short allow-list. (Raised by Codex; Claude agrees.)
| servers = mcp["mcpServers"] | ||
| if not isinstance(servers, dict) or not servers: | ||
| raise ValueError("Local MCP mode requires at least one server") | ||
| for server in servers.values(): |
There was a problem hiding this comment.
Minor (edge input, opt-in local-MCP build only). The loop checks only servers.values(), and line 159 writes the whole object back out, so server names (the mcpServers keys) are never checked. {"mcpServers":{"/Users/me/secret-server":{"command":"node"}}} passes, and that machine path lands in the packaged .mcp.json (reproduced). Suggested fix: require each name to match [A-Za-z0-9_-]+. (Raised by Claude; Codex accepted it as minor.)
|
|
||
|
|
||
| def read_file(path): | ||
| if path.is_symlink() or not path.is_file(): |
There was a problem hiding this comment.
Minor (edge input, safe in a follow-up). read_file checks only the last part of the path for a symlink. If plugins/altimate-code/.codex-plugin/ is a symlink to a directory outside the repo, the build reads the manifest from there and succeeds (reproduced). The same applies to a symlinked plugins/altimate-code/, which skips the checks on its skills/ and assets/ children. This needs a symlinked directory committed to this repo, so it is unlikely. A path.resolve().is_relative_to(repo.resolve()) check in read_file and on the two roots would close it. (Raised by Codex; Claude agrees.)
| image = ET.fromstring(archive.read(manifest["interface"][field][2:])) | ||
| self.assertEqual(image.attrib["width"], image.attrib["height"]) | ||
| self.assertGreaterEqual(int(image.attrib["width"]), 48) | ||
| for path in (REPO / "plugins/altimate-code/skills").rglob("*.md"): |
There was a problem hiding this comment.
Minor (test-only). No test protects the round-1 installer fix. This assertion compares the packaged skills with the current source, so it passes whatever the source says. Putting the round-1 get-started/SKILL.md back (where the model runs the chosen installer) leaves all 14 tests and tests/resolve-cli.sh passing (reproduced). A small assertion would guard it: get-started/SKILL.md never tells the model to run curl … | bash, keeps the "Never execute the curl pipeline" line, and keeps "Never request tokens in chat". (Raised by Codex; Claude agrees.)
|
Round-2 consensus review: comments not tied to a diff line (panel: Claude and Codex). Round-1 nits. All four are fixed:
Outside this PR. Remaining validation. The directory ZIP has still not been installed in desktop Codex. Skill execution, the directory-to-GitHub switch and the portal submission are also untested. These block submission, not merge: |
The full GitHub plugin contains lifecycle hooks and cannot be submitted unchanged to the public directory. This adds a reproducible directory ZIP while preserving the GitHub plugin and its existing hooks.
AltimateAI/altimate-frontendat revision854b86ad83d8d1a6cab576aab1dad1abfc573f78. Only outer SVG sizing and padding change; the artwork bytes are checked.DIRECTORY.md. Pin packaging actions by commit SHA.Rebased onto current main, preserving its workspace-review status feature and 1.3.0 release history. The new package release is 1.4.0.
Validation: 14 packaging tests, CLI resolver and workspace-rule hook suites, a real ZIP build and
git diff --checkpass. The new MCP and README regressions failed before the fixes. Mutation checks confirm tests detect dropped manifest stripping, changed timestamps, changed permissions, unsorted ZIP entries and CRLF artwork changes. Both manifest holders are checked in both ZIP modes.Independent review: fresh GPT-6 Astra and GPT-6.1 Sol sessions both explicitly APPROVED the full rebased PR at
1736c2767d694d3640e2f1ab168797e31b32fdc4. A supplemental Sol security pass also approved after 102 invalid-input probes, three portable configurations and 13 symlink cases. No Claude reviewer was used.Remaining validation before submission: clean desktop installation, actual skill and hook execution, the directory-to-GitHub switch, and OpenAI portal submission/acceptance. Directory installation through Codex CLI is not established. Optional MCP packaging is exercised with fixtures because the source plugin has no MCP configuration. No backend or production deployment is involved; reverting this packaging change removes the new build/setup path.
Implements the packaging and onboarding portion of AI-10711: https://altimateai.atlassian.net/browse/AI-10711